Bayesian heuristic approach to global optimization and examples

The traditional numerical analysis considers optimization algorithms which guarantee some accuracy for all functions to be optimized. This includes the exact algorithms. Limiting the maximal error requires a computational effort that in many cases increases exponentially with the size of the problem (Horst and Pardalos, 1995, Handbook of Global Optimization, Kluwer). That limits practical applications of the worst case analysis. An alternative is the average case analysis where the average error is made as small as possible (Calvin and Glynn, 1997, J. Appl. Prob., 32: 157). The average is taken over a set of functions to be optimized. The average case analysis is called the Bayesian Approach (BA) (Diaconis, 1988, Statistical Decision Theory and Related Topics, Springer; Mockus and Mockus, 1987, Theory of Optimal Decisions, Nauk, Lithuania). Application of BA to optimization of heuristics is called the Bayesian Heuristic Approach (BHA) (Mockus, 2000, A Set of Examples of Global and Discrete Optimization, Kluwer). In this paper a short presentation of the basic ideas of BHA (described in detail in Mockus (1989), Bayesian Approach to Global Optimization, Kluwer and Mockus (2000), A Set of Examples of Global and Discrete Optimization, Kluwer) is given using the knapsack problem as an example. The application potential is illustrated by the school scheduling example. In addition the new heuristic algorithm for solving a bimatrix game problem is investigated. The results ae applied while solving real life optimization problems and also as examples for distance graduate level studies of the theory of games and markets in the Internet environment. 1. Direct Bayesian approach (DBA) There are several ways of applying the BA in optimization. The Direct Bayesian Approach (DBA) is defined by fixing a prior distribution P on a set of functions f(x) and by minimizing the Bayesian risk function R(x) (DeGroot, 1970; Mockus, 1989). The risk function describes the average deviation from the global minimum. The m distribution P is regarded as a stochastic model of f(x), x [ R where f(x) might be a deterministic or a stochastic function. This is very important feature of the Bayesian approach that shows the equivalence between uncertain deterministic and the corresponding stochastic functions (Savage, 1954; Lindley, 1965; DeGroot, 1970; Fine, 1983; Zilinskas, 1986). For example, if only the values z 5 f(x ), i 5 1, . . . , n i i are known, the level of uncertainty of some deterministic function f(x), x ± x can be i represented as the conditional standard deviation s (x) of the corresponding n stochastic function f(x) 5 f(x, v) where v is a stochastic variable. In the Gaussian case (Mockus, 1989), assuming that the (n 1 1)th observation is the last one